A scalable federated learning solution for secondary care using low-cost microcomputing: privacy-preserving development and evaluation of a COVID-19 screening test in UK hospitals.

BACKGROUND: Multicentre training could reduce biases in medical artificial intelligence (AI); however, ethical, legal, and technical considerations can constrain the ability of hospitals to share data. Federated learning enables institutions to participate in algorithm development while retaining custody of their data but uptake in hospitals has been limited, possibly as deployment requires specialist software and technical expertise at each site. We previously developed an artificial intelligence-driven screening test for COVID-19 in emergency departments, known as CURIAL-Lab, which uses vital signs and blood tests that are routinely available within 1 h of a patient's arrival. Here we aimed to federate our COVID-19 screening test by developing an easy-to-use embedded system-which we introduce as full-stack federated learning-to train and evaluate machine learning models across four UK hospital groups without centralising patient data. METHODS: We supplied a Raspberry Pi 4 Model B preloaded with our federated learning software pipeline to four National Health Service (NHS) hospital groups in the UK: Oxford University Hospitals NHS Foundation Trust (OUH; through the locally linked research University, University of Oxford), University Hospitals Birmingham NHS Foundation Trust (UHB), Bedfordshire Hospitals NHS Foundation Trust (BH), and Portsmouth Hospitals University NHS Trust (PUH). OUH, PUH, and UHB participated in federated training, training a deep neural network and logistic regressor over 150 rounds to form and calibrate a global model to predict COVID-19 status, using clinical data from patients admitted before the pandemic (COVID-19-negative) and testing positive for COVID-19 during the first wave of the pandemic. We conducted a federated evaluation of the global model for admissions during the second wave of the pandemic at OUH, PUH, and externally at BH. For OUH and PUH, we additionally performed local fine-tuning of the global model using the sites' individual training data, forming a site-tuned model, and evaluated the resultant model for admissions during the second wave of the pandemic. This study included data collected between Dec 1, 2018, and March 1, 2021; the exact date ranges used varied by site. The primary outcome was overall model performance, measured as the area under the receiver operating characteristic curve (AUROC). Removable micro secure digital (microSD) storage was destroyed on study completion. FINDINGS: Clinical data from 130 941 patients (1772 COVID-19-positive), routinely collected across three hospital groups (OUH, PUH, and UHB), were included in federated training. The evaluation step included data from 32 986 patients (3549 COVID-19-positive) attending OUH, PUH, or BH during the second wave of the pandemic. Federated training of a global deep neural network classifier improved upon performance of models trained locally in terms of AUROC by a mean of 27·6% (SD 2·2): AUROC increased from 0·574 (95% CI 0·560-0·589) at OUH and 0·622 (0·608-0·637) at PUH using the locally trained models to 0·872 (0·862-0·882) at OUH and 0·876 (0·865-0·886) at PUH using the federated global model. Performance improvement was smaller for a logistic regression model, with a mean increase in AUROC of 13·9% (0·5%). During federated external evaluation at BH, AUROC for the global deep neural network model was 0·917 (0·893-0·942), with 89·7% sensitivity (83·6-93·6) and 76·6% specificity (73·9-79·1). Site-specific tuning of the global model did not significantly improve performance (change in AUROC <0·01). INTERPRETATION: We developed an embedded system for federated learning, using microcomputing to optimise for ease of deployment. We deployed full-stack federated learning across four UK hospital groups to develop a COVID-19 screening test without centralising patient data. Federation improved model performance, and the resultant global models were generalisable. Full-stack federated learning could enable hospitals to contribute to AI development at low cost and without specialist technical expertise at each site. FUNDING: The Wellcome Trust, University of Oxford Medical and Life Sciences Translational Fund.

Assessment of an institutional guideline for vancomycin dosing and identification of predictive factors associated with dose and drug trough levels.

OBJECTIVES: To evaluate the effectiveness of an antimicrobial guideline for vancomycin prescribing deployed using electronic prescribing aid and web/phone-based app. To define factors associated with guideline compliance and drug levels, and to investigate if antimicrobial dosing recommendations can be refined using routinely collected electronic healthcare record data. METHODS: We used data from Oxford University Hospitals between 01-January-2016 and 01-June-2021 and multivariable regression models to investigate factors associated with dosing compliance, drug levels and acute kidney injury (AKI). RESULTS: 3767 patients received intravenous vancomycin for ≥24 h. Compliance with recommended loading and initial maintenance doses reached 84% and 70% respectively; 72% of subsequent maintenance doses were correctly adjusted. However, only 26% first and 32% subsequent levels reached the target range, and for patients with ongoing vancomycin treatment, 55-63% achieved target levels at 5 days. Drug levels were independently higher in older patients. Incidence of AKI was low (5.7%). Model estimates were used to propose updated age, weight and eGFR specific guidelines. CONCLUSION: Despite good compliance with guidelines for vancomycin dosing, the proportion of drug levels achieving the target range remained suboptimal. Routinely collected electronic data can be used at scale to inform pharmacokinetic studies and could improve vancomycin dosing.

Probabilistic modelling of effects of antibiotics and calendar time on transmission of healthcare-associated infection.

Healthcare-associated infection and antimicrobial resistance are major concerns. However, the extent to which antibiotic exposure affects transmission and detection of infections such as MRSA is unclear. Additionally, temporal trends are typically reported in terms of changes in incidence, rather than analysing underling transmission processes. We present a data-augmented Markov chain Monte Carlo approach for inferring changing transmission parameters over time, screening test sensitivity, and the effect of antibiotics on detection and transmission. We expand a basic model to allow use of typing information when inferring sources of infections. Using simulated data, we show that the algorithms are accurate, well-calibrated and able to identify antibiotic effects in sufficiently large datasets. We apply the models to study MRSA transmission in an intensive care unit in Oxford, UK with 7924 admissions over 10 years. We find that falls in MRSA incidence over time were associated with decreases in both the number of patients admitted to the ICU colonised with MRSA and in transmission rates. In our inference model, the data were not informative about the effect of antibiotics on risk of transmission or acquisition of MRSA, a consequence of the limited number of possible transmission events in the data. Our approach has potential to be applied to a range of healthcare-associated infections and settings and could be applied to study the impact of other potential risk factors for transmission. Evidence generated could be used to direct infection control interventions.

Detection of mixed infection from bacterial whole genome sequence data allows assessment of its role in Clostridium difficile transmission.

Bacterial whole genome sequencing offers the prospect of rapid and high precision investigation of infectious disease outbreaks. Close genetic relationships between microorganisms isolated from different infected cases suggest transmission is a strong possibility, whereas transmission between cases with genetically distinct bacterial isolates can be excluded. However, undetected mixed infections-infection with ≥2 unrelated strains of the same species where only one is sequenced-potentially impairs exclusion of transmission with certainty, and may therefore limit the utility of this technique. We investigated the problem by developing a computationally efficient method for detecting mixed infection without the need for resource-intensive independent sequencing of multiple bacterial colonies. Given the relatively low density of single nucleotide polymorphisms within bacterial sequence data, direct reconstruction of mixed infection haplotypes from current short-read sequence data is not consistently possible. We therefore use a two-step maximum likelihood-based approach, assuming each sample contains up to two infecting strains. We jointly estimate the proportion of the infection arising from the dominant and minor strains, and the sequence divergence between these strains. In cases where mixed infection is confirmed, the dominant and minor haplotypes are then matched to a database of previously sequenced local isolates. We demonstrate the performance of our algorithm with in silico and in vitro mixed infection experiments, and apply it to transmission of an important healthcare-associated pathogen, Clostridium difficile. Using hospital ward movement data in a previously described stochastic transmission model, 15 pairs of cases enriched for likely transmission events associated with mixed infection were selected. Our method identified four previously undetected mixed infections, and a previously undetected transmission event, but no direct transmission between the pairs of cases under investigation. These results demonstrate that mixed infections can be detected without additional sequencing effort, and this will be important in assessing the extent of cryptic transmission in our hospitals.